Only increasing accuracy without considering uncertainty may negatively impact Deep Neural Network (DNN) decision-making and decrease its reliability. This paper proposes five combined preprocessing and post-processing methods for time-series binary classification problems that simultaneously increase the accuracy and reliability of DNN outputs applied in a 5G UAV security dataset. These techniques use DNN outputs as input parameters and process them in different ways. Two methods use a well-known Machine Learning (ML) algorithm as a complement, and the other three use only confidence values that the DNN estimates. We compare seven different metrics, such as the Expected Calibration Error (ECE), Maximum Calibration Error (MCE), Mean Confidence (MC), Mean Accuracy (MA), Normalized Negative Log Likelihood (NLL), Brier Score Loss (BSL), and Reliability Score (RS) and the tradeoffs between them to evaluate the proposed hybrid algorithms. First, we show that the eXtreme Gradient Boosting (XGB) classifier might not be reliable for binary classification under the conditions this work presents. Second, we demonstrate that at least one of the potential methods can achieve better results than the classification in the DNN softmax layer. Finally, we show that the prospective methods may improve accuracy and reliability with better uncertainty calibration based on the assumption that the RS determines the difference between MC and MA metrics, and this difference should be zero to increase reliability. For example, Method 3 presents the best RS of 0.65 even when compared to the XGB classifier, which achieves RS of 7.22.
translated by 谷歌翻译
增加分类变量的基数可能会降低ML算法的整体性能。本文介绍了一种新颖的计算预处理方法,用于转换为机器学习(ML)算法的数值变量的分类。在此方法中,我们选择并将三个分类特征转换为数值特征。首先,我们根据变量中类别的分布选择阈值参数。然后,我们使用条件概率将每个分类变量转换为两个新的数字变量,总共产生六个新的数变量。之后,我们将这六个数值送入到主成分分析(PCA)算法。接下来,我们选择主组件(PCS)的整个或部分数量。最后,通过使用十种不同的分类器应用二进制分类,我们测量了新编码器的性能,并将其与其他17个众所周知的类别编码器进行比较。所提出的技术实现了使用众所周知的网络安全NSLKDD DataSet对高基数分类变量下的曲线(AUC)下的最高性能。此外,我们定义了谐波平均指标,在火车和测试性能之间找到最佳权衡,并防止磨损和过度装备。最终,新创建的数字变量的数量很少。因此,该数据减少改善了计算处理时间,这可能减少5G未来电信网络中的处理数据。
translated by 谷歌翻译
Stellar photospheric activity is known to limit the detection and characterisation of extra-solar planets. In particular, the study of Earth-like planets around Sun-like stars requires data analysis methods that can accurately model the stellar activity phenomena affecting radial velocity (RV) measurements. Gaussian Process Regression Networks (GPRNs) offer a principled approach to the analysis of simultaneous time-series, combining the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian Processes. Using HARPS-N solar spectroscopic observations encompassing three years, we demonstrate that this framework is capable of jointly modelling RV data and traditional stellar activity indicators. Although we consider only the simplest GPRN configuration, we are able to describe the behaviour of solar RV data at least as accurately as previously published methods. We confirm the correlation between the RV and stellar activity time series reaches a maximum at separations of a few days, and find evidence of non-stationary behaviour in the time series, associated with an approaching solar activity minimum.
translated by 谷歌翻译